Add Arrcus ArcOS as a containerlab device (best-effort) - #3681
Conversation
ipspace
left a comment
There was a problem hiding this comment.
Before going into the details: I think you should change the configuration deployment method.
It looks like you're generating a bash script that takes a config and runs confd_cli to deploy it. If that's the case, you don't need Ansible -- netlab can generate bash scripts, map them to the container, and then execute them inside the container. You can also map templates into container files to create scripts (like what you have in the shell command) that are then executed.
FRR uses the "run a bash script" approach together with "let's add a shebang when needed" trick. cRPD maps a template to a container script to generate the deployment script (/config/netlab/netlab-config.sh).
More @ https://netlab.tools/dev/config/deploy/#linux-configuration-scripts
|
Next major item: we don't have per-platform integration tests (what you added in The device implementations are tested with regular integration tests (see https://netlab.tools/dev/integration-tests/). If you want to test the validation plugins, you'll find some tests in |
| global afi-safi IPV6_UNICAST | ||
| {% endif %} | ||
| {% for a in bgp.advertise|default([]) %} | ||
| {% if a.ipv4 is defined %} |
There was a problem hiding this comment.
One more generic comment -- I strongly recommend you use the same indentation trick we use in other Jinja2 templates to make the templates more readable. Unfortunately, that's the best I could come up with, and it looks like some vendors picked it up as well.
For example:
{% for x %}
{% if y is defined %}
{% endif %}
{% endfor %}
Modernize the legacy ArcOS stub into a full containerlab (arrcus_arcos) device: initial, ospf (v2/v3), bgp, isis, vrf, vlan, lag, gateway (VRRP), dhcp/relay, bfd, routing (static/prefix/policy), mpls (LDP), sr (SR-MPLS via IS-IS), srv6, vxlan and evpn (L2VNI) configuration templates, plus OpenConfig JSON validation plugins (ospf/bgp/isis/routing). Configuration is deployed with netlab native "sh" config mode: each module is rendered into /config/netlab/NN-<module>.sh with a #!/config/netlab/netlab-config.sh shebang and executed via docker exec, which hands the rendered config to the mapped wrapper templates/provider/clab/arcos/netlab-config.j2 to load through confd_cli (load merge / commit). This is the same mapped-script deploy pattern Juniper cRPD uses; no Ansible is involved in the deploy path. ansible_connection: docker is retained only for the validation/collect path, since the tested build has SSH/NETCONF/gNMI disabled. Drop the legacy Ansible deploy-config task and the unused libvirt domain template (the device is clab-only).
Install guide, caveats, platform table entry and release note for the ArcOS containerlab device, documenting the native sh-mode confd_cli deploy path and the docker-exec validation model.
|
Thanks for the review, Ivan — all three addressed. 1. Deploy method. Switched the ArcOS device from the Ansible deploy task to netlab's native container-script ( 2. Platform tests. Removed 3. Jinja2 indentation. Applied the readable nested-indentation convention (tag keyword indented with the block) across the ArcOS templates. Rebased onto current |
ipspace
left a comment
There was a problem hiding this comment.
Lots of nits and a few major showstoppers:
A) Changes to documentation (file placement, caveats)
B) IPv4/IPv6 feature parity
C) Correct configuration of IPv4-only, dual-stack and IPv6-only environments. I'm pretty sure we're not testing every potential error, so incorrect configurations could pass the integration tests just because we never thought that someone could do X.
D) Global/VRF feature parity. We expect people to use same/similar configuration for global and VRF routing protocols, and thus do not check every detail like support for areas or OSPF cost in VRF OSPF instances.
E) OSPF areas
Also, while I cannot test stuff (and wouldn't even bother with a vendor that does not have a way to download images for non-customers), I would strongly recommend you run the full set of integration tests. I'm sure some of them (like OSPF multi-area test) would fail.
| @@ -0,0 +1,54 @@ | |||
| # Installing Arrcus ArcOS | |||
There was a problem hiding this comment.
This file should be in docs/labs. The "docs/install" directory is for netlab installation, the "docs/labs" directory is for providers and devices.
| * The [**WireGuard tunnel** plugin](plugin-tunnel-wireguard) supports WireGuard tunnels on FRR. | ||
| * The [**bgp.session** plugin](plugin-bgp-session) and the [OSPF module](module-ospf) support graceful restart on Arista EOS, BIRD, FortiOS, and FRR | ||
| * The [**bgp.policy** plugin](plugin-bgp-policy) supports the **bgp.role** attribute on FRR and BIRD. | ||
| * [Arrcus ArcOS](https://arrcus.com/) is now supported (best-effort) as a native containerlab node, with configuration deployment and validation over a `docker exec` + `confd_cli` path. |
There was a problem hiding this comment.
This should be in 26.08. 26.07 is already out.
| ## Arrcus ArcOS | ||
|
|
||
| * ArcOS ships a commercial NOS image; there is no public Vagrant/Containerlab box. ArcOS runs as a **native** containerlab node (`kind: arrcus_arcos`) -- no vrnetlab packaging. Provide your own container image and tag it as in `netsim/devices/arcos.yml` (`clab.image`), or override `clab.image` in your topology. | ||
| * The tested container image boots with **SSH, NETCONF, and gNMI all disabled** (its baked-in startup config fails to load a built-in prefix-set). *netlab* bootstraps the node with the native `netlab_start_exec` group_var (containerlab's post-start `exec:`), which enables `ssh-server`, changes the factory-default admin-user password (ArcOS refuses to enable interfaces until this is done), and creates an AAA user. No custom deploy code and no SSH is used to run it. |
There was a problem hiding this comment.
I would just point to the labs/arcos.md document instead of repeating the points already made there.
| * The tested container image boots with **SSH, NETCONF, and gNMI all disabled** (its baked-in startup config fails to load a built-in prefix-set). *netlab* bootstraps the node with the native `netlab_start_exec` group_var (containerlab's post-start `exec:`), which enables `ssh-server`, changes the factory-default admin-user password (ArcOS refuses to enable interfaces until this is done), and creates an AAA user. No custom deploy code and no SSH is used to run it. | ||
| * Configuration is deployed with netlab's native containerlab **"sh" config mode**: each module is rendered into `/config/netlab/NN-<module>.sh` and executed via `docker exec`, loading config through `confd_cli` (`load merge` / `commit`) via the mapped wrapper `netsim/templates/provider/clab/arcos/netlab-config.j2` -- the same mapped-script pattern Juniper cRPD uses, with no Ansible in the deploy path. Validation/collect use `ansible_connection: docker` (`docker exec` running `confd_cli`). ArcOS ships the official `arrcus.arcos` `network_cli` collection (the project-recommended model), but every published version (2.0.13-2.0.18) hangs against this image: its cliconf/terminal plugins issue a `config terminal` / `arcos_cli` mode-switch that this build's `confd_cli` rejects, so it is not used. | ||
| * `netlab validate` reads ArcOS state over the **same** docker-exec path, running `show <path> | display json | confd_cli` and parsing the OpenConfig JSON (`netsim/validate/**/arcos.py`). This uses netlab's standard device-side show-command validation -- **no** `ansible` validation action, and no SSH/NETCONF/gNMI. | ||
| * **MTU:** the `interface <if> mtu <n>` leaf is not a pass-through -- ArcOS programs the real kernel netdev MTU (and the value it advertises in OSPF DBD packets) to `<n> - 26` on every commit. `netsim/ansible/templates/initial/arcos.j2` therefore sets the leaf to `l.mtu + 26` so ArcOS lands on netlab's intended MTU; this alone makes an ArcOS<->FRR OSPF adjacency converge, with no containerlab/core change. |
There was a problem hiding this comment.
I would just say that initial device configuration sets ArcOS to lab topology layer-3 MTU (interface.mtu) + 26.
| * Configuration is deployed with netlab's native containerlab **"sh" config mode**: each module is rendered into `/config/netlab/NN-<module>.sh` and executed via `docker exec`, loading config through `confd_cli` (`load merge` / `commit`) via the mapped wrapper `netsim/templates/provider/clab/arcos/netlab-config.j2` -- the same mapped-script pattern Juniper cRPD uses, with no Ansible in the deploy path. Validation/collect use `ansible_connection: docker` (`docker exec` running `confd_cli`). ArcOS ships the official `arrcus.arcos` `network_cli` collection (the project-recommended model), but every published version (2.0.13-2.0.18) hangs against this image: its cliconf/terminal plugins issue a `config terminal` / `arcos_cli` mode-switch that this build's `confd_cli` rejects, so it is not used. | ||
| * `netlab validate` reads ArcOS state over the **same** docker-exec path, running `show <path> | display json | confd_cli` and parsing the OpenConfig JSON (`netsim/validate/**/arcos.py`). This uses netlab's standard device-side show-command validation -- **no** `ansible` validation action, and no SSH/NETCONF/gNMI. | ||
| * **MTU:** the `interface <if> mtu <n>` leaf is not a pass-through -- ArcOS programs the real kernel netdev MTU (and the value it advertises in OSPF DBD packets) to `<n> - 26` on every commit. `netsim/ansible/templates/initial/arcos.j2` therefore sets the leaf to `l.mtu + 26` so ArcOS lands on netlab's intended MTU; this alone makes an ArcOS<->FRR OSPF adjacency converge, with no containerlab/core change. | ||
| * **Static routing validation** reads the committed route from the running-config JSON rather than an operational RIB: this virtual image does not populate an OpenConfig STATIC operational-state/RIB tree. A present, complete static-route entry proves netlab rendered it and the device accepted and committed it; kernel FIB install and forwarding were verified separately. |
There was a problem hiding this comment.
I wouldn't go into so many details. I would just say something along the lines of "netlab configures static routes and they are used for traffic forwarding, but the tested ArcOS image does not report them in OpenConfig RIB tree"
| {% endif %} | ||
| {% if routing.policy|default({}) %} | ||
| {% for p_name, p_entries in routing.policy.items() %} | ||
| routing-policy policy-definition {{ p_name }}-ipv4 |
There was a problem hiding this comment.
This seems to be IPv4-only. We can do both protocols.
| {% endfor %} | ||
| {% endif %} | ||
| {% if routing.static|default([]) %} | ||
| network-instance default protocol STATIC default |
There was a problem hiding this comment.
Yet again, seems to be the exact same configs for IPv4 and IPv6, so maybe use an inner loop.
| {% if l.type|default("") == "svi" %} | ||
| interface {{ l.ifname }} enabled true | ||
| interface {{ l.ifname }} autostate false | ||
| {% if l.ipv4 is defined and l.ipv4 %} |
There was a problem hiding this comment.
What about IPv6 over VLANs? Also: MTU?
| {% set nt = l.ospf.network_type|default('point-to-point' if l.type|default('') == 'p2p' else '') %} | ||
| {% if nt == 'point-to-point' %} | ||
| network-type POINT_TO_POINT_NETWORK | ||
| {% endif %} |
There was a problem hiding this comment.
What about cost, passive interfaces, and the like? We usually define macros in OSPF/BGP/IS-IS templates and then reuse them in VRFs
| network-instance default protocol OSPF3 p{{ pid }} | ||
| global router-id {{ ospf.router_id }} | ||
| global log-adjacency-changes LOG_ADJ_ENABLE_DETAILED | ||
| area {{ area3 }} |
There was a problem hiding this comment.
I don't see where you define other areas
Address the CHANGES_REQUESTED review (IPv4/IPv6 feature parity; correct config for IPv4-only, dual-stack and IPv6-only topologies; global vs VRF parity; OSPF areas; docs). Verified on a real arcos:8.2.1A.P2 container. - initial: one loop over netlab_interfaces configures every L3 interface (system loopback, additional loopbacks, LAG aggregates, routed ports) with IPv4 and IPv6 rendered side by side at the interface's actual prefix length -- no hardcoded /32 or /128, no hand-built loopback0 name. LAG aggregates now get IPv6 addresses too. - isis: every IPv4-AF enable (global/loopback/interface) is gated on isis.af.ipv4, so an IPv6-only box emits no IPv4 IS-IS config; redistribution renders per address family (STATIC ISIS IPV4 + IPV6). IPv6 IS-IS is single-topology -- this build has no multi-topology knob -- so cross-vendor IPv6 IS-IS does not interoperate; documented as a caveat and IS-IS MT set to unsupported in platforms.md. - routing: routing-policy definitions and static routes render per address family (inner AF loop) rather than IPv4-only. - bgp: redistribution renders per address family and now actually commits -- it needs a #COMMIT transaction split (the BGP instance must exist first) plus an explicit accept import-policy (ArcOS rejects default-import-policy ACCEPT_ROUTE on the table-connection, same as IS-IS). - ospf: multi-area support -- one area block per ospf.areas entry, each interface filed under its own ospf.area (ArcOS accepts multiple area blocks per instance); verified an ABR forms adjacencies in several areas at once. - vrf: per-VRF OSPF and IS-IS kept consistent with the global-table templates (per-interface OSPF metric/bfd/priority/timers + multi-area; dual-AF IS-IS). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015pc2sU3nSrzJNWBi1XDmEw
ArcOS accepts only letters, digits and hyphens in a hostname, so a netlab node name containing an underscore (dut_a, dut_spine, ... as used by the multi-DUT integration topologies) aborts the whole initial commit with Aborted: system hostname: Valid characters for hostnames are letters, digits and the hyphen (-). Replace _ with - the way the eos/frr/sonic/vyos initial templates already do.
The Jinja comment that documents the native-VLAN handling used whitespace-control
markers, and the leading {#- ate the newline that terminates the preceding
"switched-vlan interface-mode TRUNK" line. The next config line was appended to
it, and ConfD rejected the result:
Error: on line 1: interface swp1 ethernet switched-vlan interface-mode
TRUNKinterface swp1 ethernet switched-vlan trunk-vlans [ 502 ]
Drop the whitespace-control markers -- the comment sits on its own lines and
trim_blocks already removes them.
A routed VLAN in a trunk (vlan.mode route) never got any interface configuration:
the initial template skipped every virtual_interface, so the OSPF template was the
only one referencing the subinterface and the commit aborted with
Aborted: illegal reference 'network-instance default protocol OSPF p1 area
0.0.0.0 interface swp1.1 subinterface'
An ArcOS routed VLAN subinterface is "subinterface <id>" under the parent port,
and the subinterface ID is the 802.1Q tag: creating "interface swp1 subinterface
701" makes the device report back "vlan vlan-id 701", and the resulting interface
is named swp1.701. So name subinterfaces {ifname}.{vlan.access_id} (as eos, frr,
srlinux and the Junos devices do) and render them from the initial template's L3
interface loop, which keeps their IPv4 and IPv6 addressing in sync with every
other interface type. An untagged (native) routed VLAN keeps its addressing on the
parent port's subinterface 0 -- what ArcOS uses for untagged traffic -- and is
therefore already covered by the same loop.
ArcOS follows the OpenConfig model in which the switched-vlan container hangs off 'ethernet' on a physical port but off 'aggregation' on a LAG bond, so the VLAN template's hard-coded 'ethernet' was rejected on every bond: Error: on line 1: interface bond1 ethernet switched-vlan interface-mode TRUNK (confirmed on the device: the same line under 'aggregation' commits and shows up as "aggregation switched-vlan interface-mode TRUNK"). Pick the container from the interface type for the access, native and trunk-vlans lines alike.
The SVI branch of the VLAN template configured an IPv4 address and nothing else, so an IPv6-only or dual-stack IRB VLAN got an SVI with no subinterface at all. OSPFv3 then could not reference it: Aborted: illegal reference 'network-instance default protocol OSPF3 p1 area 0 interface vlan500 subinterface' Render IPv4 and IPv6 side by side from the same interface data, the way templates/initial/arcos.j2 already does for every other L3 interface (and use ansible.utils.ipaddr consistently with the rest of the device's templates). Also set the interface MTU leaf to l.mtu + 26 -- the same device-side compensation initial/arcos.j2 applies, because ArcOS programs the netdev and the MTU it advertises in OSPF DBD packets as (leaf - 26). Without it an SVI silently inherits the widest member port's MTU, which left an OSPF adjacency over a reduced-MTU VLAN stuck in ExStart. Verified with ospf/ospfv3/07-vlan-mtu and ospf/ospfv2/06-vlan-mtu, both of which now deploy and pass validation.
Two VRFs may legitimately use the same address on their loopbacks, but the L3VRF
network-instances and the interface bindings were only rendered by the vrf module,
which runs after the initial configuration. Both loopbacks were therefore still in
the default network-instance when their addresses were committed, and ArcOS
refused the second one:
Aborted: 'interface loopback1 subinterface 0 ipv6 address 2001:db8:c001:cafe::1':
IPv6 subnet overlaps with 'interface loopback2 subinterface 0 ipv6 address
2001:db8:c001:cafe::1 prefix-length 64'
Create the L3VRF instances and bind the interfaces from the initial template, the
way the eos/ios/junos initial templates do, so the binding is part of the same
transaction as the addressing. templates/vrf/arcos.j2 keeps the routing protocols
inside each VRF and keeps its binding loop, which is a no-op for what the initial
template already bound and still picks up the interfaces created by the modules in
between -- an SVI cannot be bound before the VLAN template creates it ("Interface
vlanN subinterface 0 does not exist").
vrf/04-multi-vrf-loopback-ipv6 and vrf/22-multi-vrf-bgp-ipv6 now deploy.
netsim/devices/arcos.yml declares routing.policy match/set capabilities, so the bgp.policy plugin accepts locpref/MED/community attributes on ArcOS BGP neighbours and then refused to configure the lab because the device had no per-device policy template: UserWarning in configs: Cannot find bgp.policy configuration template for dut/device arcos Apply the plugin-generated policies to the BGP neighbours with the OpenConfig per-AF apply-policy leaf-lists, picking the per-AF policy-definition names that templates/routing/arcos.j2 emits. Verified on the device with bgp.policy/21-locpref: the peer sees the DUT's prefixes with local preference 17 and 42 on IPv4 and IPv6 alike. Only the default network-instance is configured -- templates/bgp/arcos.j2 does not create a BGP instance inside an L3VRF (this build rejects both 'global as' and 'global afi-safi' under a VRF BGP instance), so there is no VRF neighbour to attach a policy to. bgp.weight, bgp.bandwidth, bgp.role and AS-path prepending have no ArcOS equivalent and are not declared in the device settings, so the plugin rejects them before reaching this template.
MTU is a per-port property on ArcOS: there is no per-subinterface mtu leaf, and an 'mtu' written in subinterface context silently lands on the parent interface (verified on the device -- 'interface swp1 / subinterface 701 / mtu 1600' shows up as 'interface swp1 mtu 1600'). A routed VLAN subinterface must therefore not render one: it would overwrite the parent port's MTU, and on a trunk carrying several routed VLANs the last one would win. The parent port still gets its own MTU from its own pass through the same loop, and the subinterface inherits it. No test in the suite sets an MTU on a routed VLAN link, so nothing changes today; vlan/51-vlan-routed-trunk still deploys and passes validation.
The VRF-binding loop added in "put VRF interfaces into their VRF before addressing them" had no filter, while the interface-configuration loop above it skips SVIs and other virtual interfaces. It therefore emitted a binding for an interface that does not exist yet, and ArcOS aborts the whole initial commit on it: Aborted: 'network-instance customer interface vlan1000': Interface vlan1000 which broke evpn/03-vxlan-symmetric-irb, vxlan/04-vxlan-irb-ospf and vxlan/05-vxlan-router-stick. templates/vrf/arcos.j2 already re-binds the interfaces created by the modules in between, so those interfaces just have to be left to it. Collect the interfaces the configuration loop handles in a namespace list and bind from that list instead of repeating the loop's filter, so the two can no longer drift apart.
templates/gateway/arcos.j2 renders VRRP only -- its own header says the anycast form is untested on this image -- so features.gateway.protocol must not list it. With the declaration in place netlab accepted an anycast topology and rendered an empty gateway configuration; it now refuses it at transform time (gateway/01-anycast) or drops the anycast gateway with a warning and falls back to the real SVI address (evpn/03-vxlan-symmetric-irb).
The VRRP template only ever looked at the interface IPv4 address, so a dual-stack or IPv6-only first-hop-redundancy group got no IPv6 virtual address at all. ArcOS hangs the VRRP group off the interface address of the matching address family and exposes the identical leaf set under both (verified on the device: virtual-address, priority, advertisement-interval and accept-mode all commit under ipv6 address), so both families are now rendered from one loop and an interface may be IPv4-only, IPv6-only or dual-stack. Also record why gateway.vrrp.preempt is not rendered: this image has no preempt leaf under vrrp-group in either address family.
features.bgp.activate_af was declared but the template ignored n.activate and activated whichever address family the neighbor happened to have an address for, so selective activation (bgp/20-dual-stack-activate) could not work. A dual-stack peer is two single-AF sessions on ArcOS, so the flag applies per session: the IPv4-transport session carries the IPv4 AF and the IPv6-transport session the IPv6 AF -- the same address the eos and frr templates key their neighbor activation on. A session whose address family is not activated is not configured, because ArcOS cannot bring up a neighbor with no address family.
VRRP renders and commits for both address families, but this container image ships no VRRP process: an FRR neighbour with a lower priority stays Master and counts zero advertisements received from ArcOS, and "show interface <if>" fails with "application communication failure" as soon as a VRRP group is configured on it. gateway/02-vrrp and gateway/03-vrrp-vlan therefore deploy but do not converge, in the same way SR-MPLS, MPLS/LDP and SRv6 already commit without an exercised datapath. Also record that gateway.vrrp.preempt has no leaf on this image, and add anycast gateways to the list of deliberately undeclared features.
The address-family, network and neighbor rendering moves into templates/bgp/arcos.macro.j2 so the per-VRF BGP instances can reuse it instead of reimplementing the same lines (global vs VRF parity). The AS and the router-id stay with the caller: a VRF BGP instance has no "global as" at all, and it takes its router-id from a different place in the data model. The macros take the dictionary holding the BGP instance address-family flags and iterate over [ ipv4, ipv6 ] instead of assuming IPv4 is always there, so an IPv6-only topology no longer gets a stray IPV4_UNICAST address family (and a stray STATIC->BGP IPv4 table-connection). The neighbor macro also carries the bgp.activate gating the default-instance template applied before the move: a dual-stack peer is two single-AF sessions on ArcOS, so n.activate[af] is applied per session, and a session whose only address family is not activated is not configured at all (ArcOS cannot bring up a neighbor with no address family). The gate now covers the VRF path too, where it is a no-op: netsim/modules/bgp.py sets activate[af] unconditionally on every VRF neighbor carrying that AF because the flag is not user-controllable inside a VRF. The rendered configuration is otherwise unchanged, and that is structural rather than lucky: netsim/modules/bgp.py sets node.bgp.<af> whenever any neighbor or any advertised prefix carries that AF, only ever to True, and it is not a user-settable attribute -- so for the default routing table the new address-family gate is a tautology.
netsim/devices/arcos.yml has declared vrf.bgp since the device was added, but no template ever rendered a BGP instance inside a VRF, so every PE-CE VRF BGP session was missing. Render a BGP protocol instance in each L3VRF network-instance, reusing the macros from templates/bgp/arcos.macro.j2 so the VRF instance gets the same router-id, address families, networks and (IPv4 and IPv6) neighbors as the default-instance BGP. ArcOS specifics verified on the device: * the AS number is not part of the per-VRF model (there is no "global as" inside an L3VRF); the VRF instance inherits the AS from the default instance, which is always configured when a VRF has BGP neighbours; * "global afi-safi <AF>_UNICAST" has to precede a neighbor activating that AF, but the instance, its address families and its neighbours all commit in a single transaction -- no #COMMIT split is needed; * the protocol instance tag is the VRF name, matching templates/evpn/arcos.j2 so the EVPN L3VNI configuration lands on the same instance. Route-target import/export leaking is still not rendered: it needs L3VPN_IPV4_UNICAST on the default-instance BGP, which would change every global-table session, and it is not a declared feature.
VRF route-target import/export leaking is still not supported, but the stated reason was wrong: this image does have a route-distinguisher and a route-target model (an L3VRF BGP instance accepts "global route-distinguisher", and route targets go under "global afi-safi <AF>_UNICAST rt-afi-safi L3VPN_IPV4_UNICAST route-target <rt> both" once L3VPN_IPV4_UNICAST is enabled on the default-instance BGP). Only the "no rd/RT/L3VPN model" explanation is removed; the caveat stays.
Now that templates/vrf/arcos.j2 renders a BGP instance inside an L3VRF, a bgp.policy attached to a VRF neighbor is reachable -- and was being dropped without a word: the template only ever looked at bgp.neighbors, never at vrfs.<name>.bgp.neighbors, so the policy silently did not reach the device even though templates/routing/arcos.j2 had already rendered the policy-definition it referenced. Render the per-VRF apply-policy too, through a macro shared with the default-instance path. The plugin's configuration is deployed after the vrf module, so the instance a VRF neighbor lives in already exists. The comment claiming the platform "rejects global as and global afi-safi under a VRF BGP instance" was also wrong: only global as is rejected, and this is the same wrong reason that was just removed from docs/caveats.md. The default-instance output is byte-identical.
…rted It was listed under "Genuinely unsupported, not merely untested", which the evidence contradicts: the configuration model is there. An L3VRF BGP instance accepts "global route-distinguisher" (every VRF BGP instance netlab now renders carries one), and route targets go under "global afi-safi <AF>_UNICAST rt-afi-safi L3VPN_IPV4_UNICAST route-target <rt> both". What blocks it is that the route-target leaf is rejected until L3VPN_IPV4_UNICAST is enabled on the DEFAULT-instance BGP, which changes the capabilities advertised on every global-table session. That was out of scope for the VRF BGP work, and the leaking itself has never been tested, so it stays unrendered and undeclared -- but as "not implemented", not as a platform limitation.
… templates
Both templates write "global route-distinguisher" on the same
"network-instance <vrf> protocol BGP <vrf>" instance, and evpn is deployed
after vrf (netsim/modules/evpn.yml config_after), so evpn won -- meaning a
VRF that had both an L3VNI and BGP neighbors would silently commit
<router-id>:<transit_vni> instead of netlab's vrfs.<name>.rd.
Use vdata.rd in the EVPN template as well, the way templates/vrf/eos.bgp.j2
renders "rd {{ vdata.rd }}" whether or not EVPN is in play. The two writers
now emit an identical leaf, so the committed RD matches the data model
regardless of ordering.
No test currently renders both blocks on one instance, so this changes
exactly one rendered line today: evpn_03-vxlan-symmetric-irb goes from
10.8.0.5:5042 to 65000:1.
…vdata.af
The header of templates/bgp/arcos.j2 claimed the AS and the router-id moved
into templates/bgp/arcos.macro.j2. They did not, and cannot: a VRF BGP
instance has no "global as" at all and takes its router-id from a different
place in the data model, so both stay with the caller. The macro file's own
header already said this correctly.
Also pass vdata.af through |default({}) in templates/vrf/arcos.j2, matching
templates/vrf/eos.bgp.j2. netlab's Jinja2 environment uses a chainable
undefined, so a VRF without an "af" dictionary would not raise -- it would
render an instance with no address families and no neighbors. Unreachable
today; the guard costs nothing.
The evpn template iterated vlans.items() and vrfs.items() unguarded, so any
EVPN topology whose node defines neither collection aborted at transform with
Error rendering template evpn for node dut/device arcos ... items is undefined
That broke eight integration tests outright -- evpn/01, 02, 05, 10, 11, 12, 21
and 22 -- on every branch, because a spine or a route reflector carries the EVPN
address family without owning a single VLAN or VRF.
Guard both loops with |default({}). Note the parentheses: vlans|default({}).items()
parses as vlans|default(...) applied to .items() and is a Jinja syntax error, so
the filter has to be wrapped -- (vlans|default({})).items().
…tate mechanism The substance of the previous caveat was right but two details were wrong, and an independent five-lab tiebreak established both. * The displacement is tied to the target's neighbour entry being REACHABLE, not to "the target sends anything to the device". The correlation is one-to-one, per target and per address family: on a settled lab the only discard still installed was the one whose target had no neighbour entry, while every target showing REACHABLE had none. The RIB keeps the discard as a candidate (origin STATIC, interface null0) and reports best-protocol ADJACENCY for the same /32. It is also sticky: shutting the target's interface for 90 s left the neighbour REACHABLE with no blackhole, and a hand-deleted neighbour entry is reinstated immediately. * routing/25-static-discard cannot be made reliably green on this platform by any wait value, which the previous wording did not say. For roughly the first three minutes after the DUT starts the STATIC daemon has installed nothing, so the discards do not exist; then all four appear for some 16-21 seconds while the targets are still STALE; then the targets go REACHABLE and the blackholes leave the FIB for good. Both ends of that timeline fail and only the narrow middle passes, so there is no wait value that can target it -- five fresh labs all reported 7/11. The docs/caveats.md bullet is the tiebreak's text verbatim rather than a paraphrase, so the measured timeline is recorded exactly; the support.caveats entry in netsim/devices/arcos.yml carries the same two corrections. Documentation only. features.routing.static.discard stays declared: discard works correctly for a prefix that is not directly connected.
…rything An ArcOS prefix-set is a flat member list -- the accept/reject decision lives on the policy statement that references it, not on the members. A netlab prefix list has an action per entry with first-match-wins semantics, where a 'deny' entry means "this prefix does NOT match the list". The template ignored the action and put every entry in one set, so a policy that rejects on a match of the list rejected the deny entries too. routing/10-match- prefix has been deploying that way: routing-policy defined-sets prefix-set orig_1 prefix 172.42.43.0/24 exact <- a DENY entry: must NOT match, so must not be a member prefix 172.42.42.0/24 exact Measured on the device before the fix: the DUT's BGP RIB holds neither 172.42.43.0/24 (which the topology permits) nor 172.42.42.0/24 (which it denies) -- everything the list mentions was filtered. Silent: the configuration commits and reads plausibly. Render the PERMIT entries only. The deny entries are then simply absent from the set, do not match, and fall through to the policy's later statements -- which is exactly netlab's semantics as long as no deny entry shadows a permit entry below it (refused by a quirk in the following commit). A list with no permit entries in an address family matches nothing there. That needs care: emitting an empty prefix-set would leave the statement with no condition at all, i.e. matching every prefix and applying the opposite outcome. So the set is not emitted and the statement referencing it is skipped -- visible in the IPv6 half of routing/10, where in_x1-ipv6 now carries only its catch-all. routing/10-match-prefix 20/24 -> SUCCESS 24/24, verified on the wire: the test asserts both that permitted prefixes reach x3/vx3 and that denied ones are absent. It is the only integration test using action: deny in a prefix list (routing/11 uses one in an as-path list, which ArcOS declines outright; 12 and 13 use community lists). Render sweep of all 261 topologies for -d arcos: this is the only configuration that changes, and every changed line is a removal.
Rendering the permit entries only (previous commit) is faithful as long as no
deny entry overlaps a permit entry that comes after it. When they overlap, order
is the only thing separating them, and a flat ArcOS prefix-set cannot carry
order:
- ipv4: 10.1.0.0/16
action: deny
- ipv4: 10.0.0.0/8 # 10.1.x would still match this member
10.1.0.0/16 should not match the list, but with only 10.0.0.0/8 in the set it
does. The result would be a route-map matching the wrong routes -- silent, and
visible only as traffic taking the wrong path -- so refuse it up front, in the
same form as the existing evpn_static_flooding and community_list_shape quirks.
The overlap test is on the networks themselves: if the deny and permit prefixes
do not overlap, no address can match both whatever their min/max lengths are, so
comparing networks is sufficient and errs toward accepting.
Verified both directions: the quirk fires on the topology above with the two
prefixes named in the message, and does not fire on any of the 261 integration
topologies rendered for -d arcos -- including routing/10-match-prefix, whose
deny and permit entries do not overlap.
… measured
The caveat said the ABR emits a /0 network mask for a /32 loopback and that
"IPv4 and IPv6 (OSPFv3) behave the same". Reading the neighbours' link-state
databases while triaging four near-miss tests shows the defect family is wider
than that, and that the two address families fail differently.
* OSPFv2 -- confirmed in the LSDB rather than inferred from a missing route. The
ABR's own summary LSA, read from an area-0 peer:
Link State ID: 10.8.0.1 Advertising Router: 10.8.0.1 Network Mask: /0
while the /30 transit prefixes in the same LSA set carry Network Mask: /30.
So it is specific to host routes, which the old text asserted but had not
shown.
* OSPFv3 -- NOT the same. The loopback is advertised with the interface prefix
length instead of /128: a peer installs 2001:db8:1:1::/64 where
2001:db8:1:1::1/128 is expected. The prefix is present, just not as a host
route -- a quieter failure than the IPv4 form, and one the old wording would
have led a reader to look for in the wrong place.
* In a VRF -- not mentioned at all before. A per-VRF OSPF instance acting as an
ABR shows the identical Network Mask: /0 for the VRF loopback, alongside a
correctly summarised /30, read from the VRF peer's database.
Between them these explain ospf/ospfv2/02-areas, ospf/ospfv3/06-lb-prefix,
vrf/21-multi-vrf-ospfv3 and two of the three failing checks in
vrf/11-multi-vrf-ospf. Nothing is rendered for any of it -- this is an
LSA-origination defect, not a configuration gap -- so this is documentation
only, with no rendering change.
# Conflicts: # netsim/ansible/templates/routing/arcos.j2 # netsim/devices/arcos.py
# Conflicts: # docs/caveats.md # netsim/devices/arcos.yml
Left to itself ArcOS derives the IP-VRF route target as <AS>:<L3VNI> -- 65000:5042
for a transit VNI of 5042 -- while netlab, and every other netlab EVPN device,
derives it as <AS>:<vrf-id> from vrfs.<name>.import/export, i.e. 65000:1. The two
never match, so neither side imports the other's type-5 route and symmetric IRB
does nothing at all.
Measured on the container image (arcos:8.2.1A.P2), evpn/03-vxlan-symmetric-irb:
before DUT evpn-v4-import-route-targets 65000:5042
FRR L3 VNI 5042 Import RT 65000:1 Export RT 65000:1
DUT customer VRF loc-rib: only its own two prefixes
after DUT evpn-v4-import-route-targets 65000:1
DUT customer VRF loc-rib: 172.18.2.0/24 via 10.8.0.6 <- the peer's subnet
h1 -> h4 ping starts working (a check that was failing)
Two placement details, both verified on the device and easy to get wrong:
* the RT does not live where the MAC-VRF one does. For an L2VLAN instance it is
"protocol BGP <ni> route-target <rt> both"; an L3VRF has no such node and the
value sits under the unicast AFI, in an rt-afi-safi L2VPN_EVPN container, with
a mandatory direction keyword;
* it is per address family. Setting it only under IPV4_UNICAST left the IPv6
type-5 routes still carrying the auto-derived 65000:5042, so both families are
rendered from the same netlab lists.
No feature declaration changes here: the egress half of symmetric IRB still does
not work (see the caveat correction that accompanies this), so nothing new is
claimed. This removes a wrong route target from the rendered configuration and
makes the import direction behave.
…advertised
netsim/devices/arcos.yml declared under "genuinely unsupported" that EVPN
symmetric IRB / L3VNI does not work because "route-type-5 is never originated".
Our own hardware validation had already overturned that on S8.5.1A, and it is
wrong on the container image too. Measured on evpn/03-vxlan-symmetric-irb with
arcos:8.2.1A.P2:
show overlay vni-all 5042
vni-state vni-run-state UP
vni-state vni-type ROUTED
vni-state ip-or-mac-vrf customer
default instance, L2VPN_EVPN loc-rib
65000:1:[5][0][24][172.18.0.0] ext-community [ route-target:...
65000:1:[5][0][24][172.18.1.0] encap-type:VXLAN router-mac:02:42:c0:a8:8d:6a ]
65000:1:[5][0][64][2001:db8:172::]
65000:1:[5][0][64][2001:db8:172:1::]
Four well-formed type-5 routes, one per connected subnet in the VRF. Import
works as well once the route target is written explicitly (the accompanying
template fix): the peer's subnet appears in the VRF loc-rib and a host behind
this PE starts reaching a host behind the peer.
The real gap is the egress half. The locally originated type-5 routes carry
path-types [ BEST_PATH EXPORTED_PATH ] and no rib-out entry -- they are never
advertised to the EVPN peer, so the peer never learns this PE's subnets.
"rt-afi-safi L2VPN_EVPN evpn-route-options suppress-rt5-routes false" commits
and does not change it.
Symmetric IRB stays undeclared -- nothing new is claimed -- but the recorded
reason now matches what the device does. Documentation only.
templates/bgp/arcos.j2 renders a neighbor only for an address family the neighbor activates (bgp.activate, features.bgp.activate_af), because ArcOS cannot bring up a session that has no address family. An EVPN-only session has activate.ipv4 == activate.ipv6 == false -- its address family is L2VPN_EVPN -- so the whole neighbor block was skipped there, and templates/evpn/arcos.j2 created the neighbor with no AS at all. The device accepted it and reported: neighbor 10.8.0.3 state peer-as 0 state session-state INCOMPLETE with the peer sitting in Active forever (evpn/12-vxlan-ibgp-ebgp). Emitting peer-as alongside the L2VPN_EVPN activation fixes it; it is idempotent when the BGP template also rendered the neighbor, being the same leaf with the same value. Verified on arcos:8.2.1A.P2 -- the EVPN session to the other ArcOS node goes from INCOMPLETE to: neighbor 10.8.0.3 state peer-as 65000 state session-state ESTABLISHED
netlab's replace_global_as (default true) means the local AS REPLACES the global
one in the AS path rather than being prepended to it -- "no-prepend replace-as"
on FRR, and two identically named leaves under ebgp-local-as here. The template
rendered only "local-as", so the global AS stayed in the path.
That is invisible until a peer's own AS equals the global one, which is exactly
what an IBGP-over-EBGP design does. In evpn/12-vxlan-ibgp-ebgp the spine
(global AS 65000, local-as 65100) advertised its loopback to an FRR leaf whose
real AS is also 65000; FRR discarded the update as an AS-path loop -- silently,
since a loop is dropped before policy, so it showed "0 accepted" with
"Inbound filtered: 0" -- the leaf never got a route to the spine loopback, and
the EVPN session running over that loopback stayed Active indefinitely.
The other ArcOS leaf accepted the same update, which is why only the FRR peer
was affected and why this looked like an EVPN problem rather than an AS-path one.
Verified on arcos:8.2.1A.P2, adding both leaves on the live spine:
before l2: show ip bgp -> only its own 10.8.0.4/32
l2: ip route get 10.8.0.5 -> Network unreachable
l2: EVPN neighbor 10.8.0.5 -> Active, never up
after l2: 10.8.0.5/32 10.10.0.5 0 65100 i <- AS path is the local AS
l2: EVPN neighbor 10.8.0.5 -> up 00:00:46, 18 messages received
Gated on replace_global_as so a topology that asks for prepend semantics still
gets them.
70f72e9 corrected the claim in netsim/devices/arcos.yml but two copies survived, which is the same way three copies of the community claim survived on an earlier branch: * templates/evpn/arcos.j2 still carried a "KNOWN NOT FUNCTIONAL ... ZERO route-type 5 (IP Prefix) routes are ever originated" block; * docs/caveats.md still listed "a route-type-5 IP-prefix route is never originated" under genuinely unsupported. Both now say what was measured: type-5 routes are originated (one per connected subnet, RD-qualified, encap-type:VXLAN, per-VTEP router-mac, with the L3VNI reporting vni-type ROUTED) and can be imported, and the gap is that they are never advertised to the peer -- no rib-out entry, and suppress-rt5-routes false does not change it. The template comment also records why the old measurement said zero: it predates the VRF BGP instance carrying its connected subnets, so there was nothing to originate. The same docs/caveats.md sentence also still claimed "redistribution sources other than static", which the OSPF/IS-IS-into-BGP work made stale. Corrected in the same edit rather than left in a line being rewritten: redistribution into an IGP is unsupported, redistribution into BGP works for connected, static, OSPF and IS-IS. Documentation only; no feature or template behaviour change.
# Conflicts: # docs/caveats.md # netsim/devices/arcos.py # netsim/devices/arcos.yml
# Conflicts: # docs/caveats.md # netsim/devices/arcos.yml
…easurement dbfcb85 narrowed the leaking claim in netsim/devices/arcos.yml and in templates/vrf/arcos.j2 but never touched docs/caveats.md, so the user-facing document still carried the pre-measurement text. The two files contradicted each other: the device file says "measured as unsupported, not merely untested" while caveats.md said "the leaking itself has never been tested". caveats.md now says what was measured: * the RT/L3VPN form -- the one netlab's vrfs.<name>.import/export model maps onto -- was measured, not assumed. Its export half completes: with a per-VRF label-allocation-mode and an MPLS reserved-label-block using BGP_COMMON, the VRF reaches "Export ready" and the per-VRF exported-rib fills with correctly targeted, labelled VPN routes. The label block is the step everything waits on, so the two intermediate states and the LibLabelAlloc error are recorded for whoever retries this; * the import half never fires for a locally exported route -- default-instance L3VPN loc-rib empty, every VRF at "paths-imported 0" while reporting "Import ready" with the correct import route targets -- so RT-based leaking here needs a remote PE, which a single-device leaking topology has not got; * ArcOS's OTHER leaking mechanism, which caveats.md did not mention at all: "static-route <pfx> next-hop-index N next-network-instance-name <vrf>" works and forwards (3/3 between hosts in two VRFs, with a third VRF still isolated) but takes host routes only, so it cannot carry netlab's prefix-based lists. That is the counter-example a reader would otherwise find and conclude the caveat was wrong. Also corrects a claim this document made that later measurement disproved: enabling L3VPN_IPV4_UNICAST on the default-instance BGP does NOT change the capabilities advertised on existing global-table sessions. ArcOS activates address families per neighbour; on a live bgp/01-ebgp-session lab both sessions stayed Established with no reset and negotiated no additional address family after the instance-level AF was added. Documentation only.
|
Thanks Ivan — I took the integration-test recommendation seriously and ran the full set. You were right about multi-area OSPF. Results: 25 → 77 of 181 tests passing, measured before and after with the same harness and stock addressing, no regressions. Every deploy failure eliminated (24 → 0). On (E), OSPF areas — your prediction was correct, and here is the mechanism. On (B) and (D), IPv4/IPv6 and global/VRF parity — both axes were swept, and gaps were either implemented or declared honestly. Several capabilities that were previously declared unsupported turned out to work, and are now implemented and verified on the device rather than from
Others were removed from the feature list because the platform genuinely cannot do them. On (A) — the per-platform integration tests are gone, as you asked. The documentation placement I have not fixed yet; it will be in the next push. On (C) — dual-stack is covered by the suite and passes. IPv6-only and IPv4-only topologies I have not systematically verified, so I would rather say that than claim it. One process question. Addressing all of this has grown the branch considerably — it now touches 17 template modules. Would you prefer it staged as a series of smaller dependent PRs (device skeleton → IGP → BGP/routing-policy → L2 → overlay → remaining modules), or kept as one? Happy either way; I would rather ask than force-push a restructure onto a review you are part-way through. |
…device docs/install/ holds general installation documentation (clone, cloud, linux, ubuntu); per-device pages live in docs/labs/ -- arubacx, asav, ceos and the rest are all there. The ArcOS page was in the wrong directory. Also adds the clab.build key pointing at the page, matching the fourteen other devices that carry one, so the device definition links to its own documentation.
netlab only sets ospf.af.<family> for the address families a node actually
has, so on an IPv6-only node ospf.af.ipv4 is undefined. The guard read
{% if ospf.router_id is defined and ospf.af.ipv4|default(true) %}
so the default let it through, the header rendered, and the area loop
underneath correctly produced nothing -- leaving a stranded OSPFv2 process
with a router ID and no areas and no interfaces:
network-instance default protocol OSPF p1
global router-id 10.0.0.1
global log-adjacency-changes LOG_ADJ_ENABLE_DETAILED
The device does not object. Deployed on arcos:8.2.1A.P2, `show running-config
network-instance default` keeps the empty stanza with no error and no warning,
so nothing in the render-commit path flags it, and whether an integration test
notices depends entirely on whether that test happens to check OSPFv2 state.
This was the only |default(true) address-family gate in the ArcOS templates.
The OSPFv3 guard sixteen lines below, all six guards in isis/arcos.j2 and the
OSPF/IS-IS guards in vrf/arcos.j2 already use |default(false).
Verified:
* Render sweep, 47 topologies x 3 address-family modes, base vs branch. 28
combinations differ and all 28 have the identical shape -- the same three
lines removed, nothing added anywhere:
- network-instance default protocol OSPF p1
- global router-id 10.0.0.1
- global log-adjacency-changes LOG_ADJ_ENABLE_DETAILED
No IPv4-only render changed at all. Every differing "dual-stack" combination
is an ospfv3 test, and those are natively IPv6-only
(tests/integration/ospf/ospfv3/topology-defaults.yml sets ipv4: False on all
three pools). The genuine dual-stack topology, ospfv3/08-dual-stack, renders
byte-identically and keeps its full OSPFv2 block.
* On the device: IPv6-only ospfv2/01-network and IPv6-only
bgp/02-ibgp-ebgp-session now show only `protocol OSPF3 p1` (the second case
confirms this was never confined to the OSPF tests); IPv4-only
ospfv2/01-network still shows `protocol OSPF p1` with area 0.0.0.0 and all
its interfaces; dual-stack ospfv3/08-dual-stack still shows both instances.
* OSPF integration suite (35 tests, ospfv2 + ospfv3), base and branch: 34 of 35
identical. The one that moved, ospfv3/42-bfd, is unstable on both sides under
the 3-run protocol -- base PASS/FAIL/FAIL, branch FAIL/PASS/PASS -- so it is
UNSTABLE rather than a regression, and the base run's PASS was luck.
Does that mean there are zero tests that fail in config or validate phase, or just that the configurations generated through the templates are not invalid?
I'm guessing this hints at what the above answer might be ;) Anyway, would love to know the exact details.
There are still several unresolved documentation comments. Even if you feel like the whole thing is ready to be deployed, please address them.
There are IPv4-only and IPv6-only BGP/OSPF tests; it's just that in many cases, we don't check whether the tested device has also enabled the other address family.
Keep it as one PR. Once the documentation is polished, I'll merge it (I can't test it anyway 🤷🏻♂️) |
Good question, and the honest answer is "the first for config, definitely not for validate". The full breakdown of the 181:
So: zero deploy failures — every topology that transforms gets a configuration ArcOS accepts and commits. But 27 tests still fail validation, which is the number I should have led with rather than the 77. It went from 24 config-phase failures to none, and from 45 validation failures to 27.
The ABR accepts interfaces in several areas, forms adjacencies in each, and floods correct inter-area Type-3 summaries for transit prefixes. What it gets wrong is a /32 loopback host route: read from a neighbour's database, the ABR's own summary LSA for its and the neighbour installs it as a OSPFv3 gets it wrong differently — it advertises the loopback with the interface prefix length instead of /128, so the prefix is present but not as a host route. That is the quieter of the two failures. The same thing happens inside a VRF: a per-VRF instance acting as ABR shows the identical Net effect: a loopback placed in a non-backbone area is unreachable from other areas; loopbacks in area 0 and all inter-area transit prefixes are fine. It is written up in
That is exactly the hole, and there was one on our side. Rendering ten topologies across the modules in IPv4-only / dual-stack / IPv6-only turned up an IPv6-only node emitting an empty OSPFv2 instance — router-id and log-adjacency-changes, no area, no interfaces, on a node whose addresses are all IPv6. ArcOS accepts and keeps it without complaint, and no test catches it: the render succeeds, the commit succeeds, and the IPv6-only topologies that do fail, fail on their own IPv4 assumptions and would fail identically with the block absent. Cause was I am now sweeping the rest of the PR for the same class, per your note that it probably applies more widely, and working through the outstanding documentation comments. |
…o it Review: "add (build-arcos)= or (labs-arcos)= anchor so you do not have to hyperlink to a file name" and "add a link to the anchor at the top of labs/arcos.md" from docs/platforms.md. Uses (build-arcos), matching (build-frr) / (build-eos) / (build-ceos), and links the containerlab column of the provider-support table the same way the vEOS row above it does.
Review: "this should be in 26.08. 26.07 is already out." docs/release/26.08.md is taken verbatim from origin/dev (this branch predates it) with the ArcOS entry added under New Functionality and nothing else changed -- diffing the file against origin/dev shows only that one bullet. The toctree line in docs/release.md is upstream's own, added here so the new file is reachable while the branch is still behind dev. The entry itself is shortened to a single sentence pointing at the two documents that carry the detail, rather than restating the deployment path.
Review: "I try to keep caveats short", "I would just point to the labs/arcos.md document instead of repeating the points already made there", and "I would skip this. Things don't work, they are not implemented, they are not enabled in features, so you can't use them. Let's move on." The section was 57 lines and 3973 words -- more than five times the next longest device in this file (Fortinet FortiOS, 771 words) and eight times Arista EOS. It is now 33 lines and 343 words. The cut follows one rule: a caveat earns its place only if _netlab_ accepts the topology and the result is silently wrong or degraded. Everything a user cannot reach, because the feature is undeclared or refused during transformation with its own message, is gone -- OSPF default origination, inter-VRF route leaking, EVPN symmetric IRB / L3VNI, the BGP link-bandwidth community, per-member actions in community lists, the STP restrictions, the EVPN-with-static-VXLAN combination, and the "genuinely unsupported" list. The VRRP file-descriptor bootstrap is gone too: _netlab_ does it automatically, so it is not something the reader has to act on. Installation, first-boot bootstrap, deployment mechanism and validation are no longer restated here; they are in labs/arcos.md, which this section now links to. Deployment is described in one clause, as suggested. The VLAN/SVI model note moved into labs/arcos.md, where it is reference material rather than a caveat. Kept, and shortened, with the wording from the review where it was given: the MTU + 26 adjustment, static routes missing from the OpenConfig RIB tree, the SR-MPLS / MPLS-LDP / SRv6 dataplane not working in the container, the ~70 second BGP startup delay, the OSPF ABR mis-summarizing loopback host routes, IS-IS IPv6 being single-topology, IPv4 VRRP not interoperating, static VXLAN flooding not being able to originate BUM traffic, and discard static routes being displaced by a connected host route.
The same defect as 9234216, in the VRF template. The per-VRF OSPFv2 block was guarded only by `vdata.ospf is defined`, with no address-family test, while the per-VRF OSPFv3 block sixteen lines below already had `vdata.ospf.af.ipv6`. netlab sets vrf.ospf.af.<family> only for the families the VRF has, so an IPv6-only VRF got a stranded instance -- a router ID and no areas and no interfaces, the inner `if l.ipv4 is defined` loop correctly producing nothing: network-instance customer protocol OSPF p-customer global router-id 10.0.0.1 Found by rendering every module ArcOS declares in IPv4-only, dual-stack and IPv6-only and reading the output, after review noted the address-family question "probably applies to the rest of the PR as well". `vdata.ospf.af` read out of the rendered VRF data is `{'ipv4': True}` where the VRF has IPv4 and `{'ipv6': True}` where it does not, so `|default(false)` can only suppress a VRF netlab has already said has no IPv4 OSPF. Verified: * Render sweep, 88 topologies x 3 address-family modes, base vs branch: 16 combinations differ, all of them the same two-line removal, zero lines added anywhere -- 28 `network-instance <vrf> protocol OSPF p-<vrf>` headers and their 28 `global router-id` lines, across the red, blue, green, common and customer VRFs of nine topologies. * Three of those combinations are in the v4-only and dual-stack columns, which looks wrong and is not: `vrf/21-multi-vrf-ospfv3` and `vrf/23-multi-vrf-mixed-ipv6` load `defaults-ipv6-only.yml`, which pins the blue VRF to IPv6-only pools whatever the rest of the topology does. Its `ospf.af` is `{'ipv6': True}` in every column. * Mirror case, in the same rendered file: with `vrf/21` in the IPv4-only column, `red` (`ospf.af {'ipv4': True}`) keeps `network-instance red protocol OSPF p-red` while `blue` keeps only `protocol OSPF3 p-blue`. * OSPF, VRF and vlan/52 integration tests, 54 in total, base and branch: all 54 identical.
# Conflicts: # docs/caveats.md # docs/release/26.08.md
|
Pushed. This is the first time the branch has been updated since your review — you had only ever seen the initial two commits, so there are 90 on top of that, plus a merge of current Documentation (the merge condition)
The address-family sweepYou said it probably applied to the rest of the PR, and you were right — there was a second instance. I rendered every module the device declares in IPv4-only, dual-stack and IPv6-only: 211 topologies, 633 combinations, and read the rendered configuration rather than only scoring tests. Two shapes were hunted: "A block emitted for a family the node does not have" — one more defect. Verified the same way as before: 88 topologies × 3 modes, base vs branch, gives 16 differing combinations and all 16 are the identical two-line removal with zero lines added anywhere — 28 stanza headers and their 28 router-id lines, across five VRF names in nine topologies. All 54 OSPF/VRF/VLAN integration tests render identically before and after. "A block emitted for only one family when both are present" — zero hits. Nowhere in the 633 combinations does a dual-stack render carry fewer constructs of a family than the corresponding single-family render. That is the shape your On your five specific pointers: four of them were already addressed by One thing I did not change, because I do not think it is mine to decide
Merge of current
|
Add Arrcus ArcOS as a containerlab device (best-effort)
This PR adds Arrcus ArcOS as a netlab device. ArcOS runs as a native
containerlab node (
kind: arrcus_arcos) — there is no vrnetlab packaging and noVagrant box. Support level is best-effort.
Connection and validation model — no core change
ArcOS ships the official
arrcus.arcosAnsible collection (the project-recommendednetwork_climodel), but every published version (2.0.13–2.0.18) hangs against thetested container image: its cliconf/terminal plugins issue a
config terminal/arcos_climode-switch that this build'sconfd_clirejects. Configuration istherefore deployed over Ansible's built-in
dockerconnection plugin (docker execrunningconfd_cliload merge/commit) — the same patternsonic_clabalready uses. The device keeps a commented-outnetwork_cliblock toswitch to once a working collection version ships.
netlab validatereads device state over the same docker-exec path(
show <path> | display json | confd_cli, parsed as OpenConfig JSON bynetsim/validate/**/arcos.py). This is netlab's standard device-side show-commandvalidation — no
ansiblevalidation action and no change to any core file.netsim/cli/connect.py,netsim/cli/validate.py,netsim/augment/validate.py,netsim/attributes.yml,netsim/templates/provider/clab/clab.j2, andtests.pyareuntouched (byte-identical to
dev). This is a pure device addition.The tested image boots with SSH/NETCONF/gNMI disabled and refuses to enable
interfaces until the factory admin-user password is changed; both are handled with
the native
netlab_start_execgroup_var (containerlab's post-startexec:) — nocustom deploy code.
Module coverage
initial,ospf(v2/v3),bgp,isis,vrf(+ospf/isis/bgp),vlan,lag,gateway(VRRP),dhcp/relay,bfd,routing(static/prefix-set/policy),mpls(LDP),sr(SR-MPLS via IS-IS),srv6,vxlan, andevpn(L2VNI).Honest limitations are documented in
docs/caveats.md(e.g. EVPN symmetric IRB/L3VNI,VRF route-target leaking, and OSPFv2 area authentication are genuinely unsupported and
not declared; MPLS/SR/SRv6 bring up the control plane only on the native container;
DHCP relay renders but its datapath is not exercised on a clab-only host, so ArcOS is
omitted from the DHCP support table).
Verification
Verified on real ArcOS containers (image
arcos:8.2.1A.P2), containerlab provider,against this branch:
02-ospf— OSPFv2 adjacency03-bgp— eBGP session + prefixes04-isis— IS-IS adjacency + reachability05-routing— static routeproto static) installed and loopback-to-loopback ping 0% lossospf+bfd+vrfnetlab initialcommits every module cleanly (idempotent on re-run) andnetlab collectretrieves the running configuration. The rendered CLI and the show/get pathsused by the validators were cross-checked against the vendor ArcOS S8.5.1A CLI
reference.
Files
netsim/devices/arcos.ymlnetsim/ansible/tasks/{deploy-config,fetch-config}/arcos.ymlnetsim/ansible/templates/*/arcos.j2(16 modules)netsim/validate/arcos.py+netsim/validate/{ospf,bgp,isis,routing}/arcos.py(adds the
netsim/validate/routing/package)tests/integration/platform/arcos/docs/platforms.md,docs/caveats.md,docs/install/arcos.md,docs/release/26.07.mdThis PR also removes
netsim/templates/provider/libvirt/arcos-domain.j2, an orphanedlibvirt domain template from a much older, never-completed ArcOS device (no
netsim/devices/arcos.ymlever existed for it); the new device is containerlab-only.ArcOS is a commercial NOS, so users supply their own container image (as with
sros/dellos10/arubacx).